1. Field of the Invention
The present invention relates to an environment monitoring system, and more particularly to an environment monitoring system for picking up the environment (road image) of one""s own vehicle by an image pick-up means such as a camera installed on the vehicle such as a motor car, detecting a vehicle approaching from the environment of one""s own running vehicle using the road image picked up and giving a warning to a driver.
2. Description of the Related Art
For example, when a driver of a vehicle running on one-side two lanes of e.g. a speed way intends to change his own vehicle lane, if he changes the lane while he misses a vehicle which catches up with his own vehicle on another lane at a higher speed than his own vehicle from the diagonal-rear direction, there is strong possibility of a serious accident. Therefore, it is desired that the driver surely notices or recognizes other vehicles in the environment.
When the following vehicle on the same lane abruptly approaches his own vehicle from the rear, if the driver of his own vehicle makes abrupt braking, there is possibility of bumping-into-the back.
When another vehicle runs forward on the same lane at a lower speed than that of his own vehicle, if a driver dozes, there is a danger of bumping-into-the back.
A technique for avoiding the danger as described above has been proposed as an environment monitoring system for a vehicle disclosed in JP-A-7-50769. Now referring to FIGS. 10A-10D, an explanation will be given of this environment monitoring system.
FIGS. 10A-10C are views for explaining a change in a rear background image acquired by a video camera 1. FIG. 10A shows a status inclusive of one""s own vehicle. FIG. 10B shows an image picked up by a video camera 1 at timing t in an environment of one""s own vehicle. FIG. 10C shows an image picked up at timing t+xcex94t.
Now it is assumed that one""s own vehicle is running straight on a flat road. The road sign and building residing in the rear of one""s own vehicle in FIG. 10A are observed as images shown in FIGS. 10B and 10C at timings t and t+xcex94t, respectively. Coupling the corresponding points in these two images provides speed vectors as shown in FIG. 10D. The are referred to as xe2x80x9coptical flowsxe2x80x9d. Where a following vehicle approaches, the directions of the vectors in the optical flows in FIG. 10D are contrary. Using the optical flows, the conventional environment monitoring system monitors the relationship between one""s own vehicle and the following vehicle or another vehicle running on an adjacent lane to detect the other vehicle approaching one""s own vehicle, thereby giving a warning to a driver.
In some prior arts, a technique of searching corresponding points using two cameras is adopted. Specifically, an edge point Pa of an object is detected from a luminance difference between the adjacent pixels on the image picked up by a camera. A point Pb of the image picked up by another camera corresponding to the detected edge point is detected. The position of an approaching vehicle is acquired by the pixel coordinates of Pa and Pb. On the basis of the position of the approaching vehicle acquired, the pertinent driver is given a warning of the existence of the vehicle approaching his own vehicle.
However, where it is difficult to recognize the status of the environment using the image picked up by a camera owing to fog or strong rain, the above conventional environment monitoring system cannot detect the approaching vehicle through image processing.
In the environment monitoring system using optical flows, in order to acquire the optical flows, for example, a xe2x80x9ccorrelative techniquexe2x80x9d of detecting the corresponding points between two frames of images"" has been adopted. Referring to FIGS. 11A and 11B, an explanation will be given of the correlative technique. First, on an image picked-up at timing t, a window W1 is set for a pixel Q at issue (FIG. 11A). Next, on the image picked up at timing t+xcex94t, while the window corresponding to the pixel Q is moved on the entire region of the image picked-up, the absolute value in the luminance difference between the corresponding pixels at timings t and t+xcex94t is acquired. This operation will be carried out for each of the pixels constituting the image. The point in the window W2 when the sum of the luminance differences for the respective pixels is minimum is defined as a corresponding point, i.e. corresponding pixel R. The vector QR thus created is obtained as an optical flow (FIG. 11B).
The correlative technique has the following disadvantage. The optical flow is detected on the basis of the luminance of each of the pixels constituting the window set within each of two frames of images picked up by a camera 1. Therefore, while a vehicle runs in bad weather such as fog or strong rain, the entire image picked-up will be whitish and hence the luminance of each of the corresponding pixels is approximately equal on the entire image. This makes it difficult to detect correct optical flows, and hence impossible to detect the approaching vehicle.
Also in the environment monitoring system using two cameras, the corresponding Pa and Pb between the two frames of images picked up by the two cameras are detected by means of the correlative technique. Therefore, while a vehicle runs in bad weather such as fog or strong rain, it is difficult to detect the corresponding points Pa and Pb, thereby making it impossible to detect another approaching vehicle.
When the environmental condition cannot be recognized using the images picked up by the cameras, the environment monitoring system cannot detect the approaching vehicle, and cannot give a warning even if there is the approaching vehicle. In this case, where a driver continues to run his own vehicle without knowing this fact, he may change the lane with the realization that there is no approaching vehicle. This may lead to the collision of his own vehicle with the approaching vehicle in the adjacent lane. In this way, it is very dangerous for a driver to run his own vehicle with no realization that the approaching vehicle cannot be detected.
An object of the present invention is to provide an environment monitoring system which permits one""s own vehicle to run safely even if an approaching vehicle cannot be detected.
In order to attain the above object, as shown in FIG. 1A, an environment monitoring system for a vehicle according to the invention comprises:
image-pickup means 1 loaded on the vehicle for picking up an environment of the vehicle to acquire an image;
image-processing means 3a-1 for processing the image to detect an approaching vehicle;
danger deciding means 3a-2 for deciding danger for the approaching vehicle;
decision means 3a-3 for deciding whether or not the approaching vehicle can be detected by means of the image processing means 3a-1; and
warning means 5 for giving warning when it is decided that the approaching vehicle cannot be detected by means of the image processing means.
In this configuration, a driver can notice that the warning means indicates that the approaching vehicle cannot be detected by the image processing means.
Preferably, the decision means decides xe2x80x9cNOxe2x80x9d if prescribed information is not acquired at any timing from the image picked up by the image pick-up means. Therefore, it is possible to decide directly from the image picked up by the image pick-up means whether or not the approaching vehicle can be detected by the image processing means.
Preferably, as shown in FIG. 1B, the prescribed information is a first image of an object at a prescribed position in the image picked up by the pick-up means when the approaching vehicle can be detected by means of the image processing means; and
the deciding means has an image storing means 3b-1 for storing the first image, the deciding means decides xe2x80x9cNOxe2x80x9d if similarity between the first image stored in the image storing means and a second image of the object at the prescribed position in the image picked up at any timing by the image pick-up means is lower than a prescribed value.
In this configuration, the decision means decides on the basis of similarity between the first image and the second image so that the image processing for only the second image to be performed for the decision can be reduced.
Preferably, the image processing means comprises: a white line detecting means 3a-11 for detecting a white line from the image picked up at any timing by the image pickup means. The prescribed information is a normal detection degree of a white line to be acquired from the image picked up by the pick-up means when the approaching vehicle can be detected by means of the image processing means. The deciding means 3a-3 comprises includes means 3a-31 for computing an optional detection degree of the white line detected by the white line detecting means, and decides xe2x80x9cNOxe2x80x9d when the optional detecting degree does not satisfy the normal detection degree. In this configuration, even when the environment cannot be partially picked up by the image pick-up means because of stain applied to the image pick-up means, it is not decided that the prescribed information has not been obtained although it can be obtained. Therefore, it can be detected whether or not the prescribed information has been acquired from the image picked up by the image pick-up means.
Preferably, the environment monitoring system, as shown in FIG. 1C, further comprises a light-on detecting means 6 for detecting turn-on of a light of a vehicle running in the environment. The prescribed information includes the normal detection degree of the white line and a normal image from the light of another vehicle running in the environment in the image picked up by the pick-up means when the approaching vehicle can be detected by means of the image processing means. The deciding means comprises includes light image detecting means 3a-32 for detecting an optional image from the light of another vehicle running in the environment in the image picked up by the pick-up means at any timing; and when the light-on detecting means detects xe2x80x9cONxe2x80x9d of the light, the deciding means decides xe2x80x9cNOxe2x80x9d if the optional image is an image not obtained when the approaching vehicle can be detected by means of the image processing means, and when the light-on detecting means does not detect xe2x80x9cONxe2x80x9d of the light, it decides the normal detection degree of the white line as the prescribed information.
In this configuration, in the nighttime while the headlight is xe2x80x9cONxe2x80x9d, the normal image obtained by the light is used as the prescribed information, and in the daytime while the headlight is xe2x80x9cOFFxe2x80x9d, the degree of normal detection of the white line is used as the prescribed information. Therefore, in both daytime and nighttime, it can be detected by the decision means 3a-3 whether or not the approaching vehicle can be detected by the image processing means 3a-1.
Preferably, as shown in FIG. 1D, using a wiper usage detecting means whether or not a wiper is used, the deciding means decides xe2x80x9cNOxe2x80x9d when the wiper usage detecting means detects that the wiper is used. In this configuration, it can be decided indirectly without performing image processing whether or not the approaching vehicle can be detected.
Preferably, as shown in FIG. 1D, the image processing means includes an optical flow detecting means for detecting, as an optical flow, movement of the corresponding points in two images obtained at two timings by the image pick-up means; and the danger deciding means decides danger by detecting the vehicle approaching one""s own vehicle on the basis of magnitude of the optical flow detected by the optical flow detecting means.
In this configuration, without using two image pick-up means, it is only necessary to decide whether or not the prescribed information can be acquired from the image picked up by a single image pick-up means at any timing.
Preferably, as shown in FIG. 1C, the image processing means comprises:
the white line detecting means for detecting a pair of white lines located on both sides of one""s own lane on which one""s own vehicle should run;
optical flow detecting means for setting a crossing point of extended lines of the pair of white lines as an FOE (focus of expansion) and detecting, as an optical flow, movement of the corresponding points in two images obtained at two timings by the image pick-up means; and
the danger deciding means decides danger by detecting the vehicle approaching one""s own vehicle on the basis of magnitude of the optical flow detected by the optical flow detecting means.
In this configuration, the white line detecting means for detecting the white line to detect the optical flow can be used to decide whether or not the approaching vehicle can be detected.
Preferably, as shown in FIG. 1A, the environment monitoring system comprises image processing stopping means for stopping the image processing means from detecting the approaching vehicle when the deciding means decides that the approaching vehicle cannot be detected by the image processing means.
In this configuration, the driver is not misled by the incorrect detection of the approaching vehicle.
The above and other objects and features of the present invention will be more apparent from the following description taken in conjunction with the accompanying drawings.